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Abstract 

We study the limiting behavior of a random dynamic system driven by a stochas- 
tic chain. Our main interest is in the chains that are not necessarily ergodic 
but rather decomposable into ergodic classes. To investigate the conditions un- 
der which the ergodic classes of a model can be identified, we introduce and study 
an ^-approximation and infinite flow graph of the model. We show that the l\- 
approximations of random chains preserve certain limiting behavior. Using the 
^-approximations, we show how the connectivity of the infinite flow graph is re- 
lated to the structure of the ergodic groups of the model. Our main result of this 
paper provides conditions under which the ergodicity groups of the model can be 
identified by considering the connected components in the infinite flow graph. We 
provide two applications of our main result to random networks, namely broadcast 
over time- varying networks and networks with random link failure. 

keywords: Ergodicity, ergodicity classes, infinite flow, product of random matrices. 

1 Introduction 

The dynamic systems driven by stochastic matrices have found their use in many prob- 
lems arising in decentralized communication [51 El EHl E] , decentralized control [El [271 EH 
1201 |2T] . distributed optimization [381 EHl E01 EHl [17], and information diffusion in social 
networks [HI [I]. In many of these applications, the ergodicity plays a central role in 
ensuring that the local "agent" information diffuses eventually over the entire network of 
agents. The conditions under which the ergodicity happens have been subject of some 
recent studies [3H ESj . However, the limiting behavior of the dynamics driven by time- 
varying chains has not been studied much for the case when the ergodicity is absent. In 
this context, a notable work is [19] where the limiting behavior of the Hegselmann-Krause 
model [15] for opinion dynamics has been studied. The stability of this (deterministic) 
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model has been established assuming certain conditions on the model. Studying the sta- 
bility of this model is important for understanding group formation in both deterministic 
and random time- varying networks, such as multiple-leaders/multiple- followers networked 
systems. 

The main objective of this paper is to investigate the limiting behavior of the linear 
dynamics driven by random independent chains of stochastic matrices in the absence of 
ergodicity. Our goal is to study the conditions under which the ergodic groups are formed 
and to characterize these groups. To do so, we introduce an ^-approximation and the 
infinite flow graph of a random model, and we study the properties of these objects. 
Using the established properties, we extend the main result of the previous work in [36] 
to a broader class of independent random models. We then proceed to show that for 
certain random models, although the ergodicity might not happen, the dynamics of the 
model still converges and partial ergodicity happens almost surely. In other words, under 
certain conditions, ergodic groups are formed and we characterize these groups through 
the connected components of the infinite flow graph. We then apply the results to a 
broadcast-gossip algorithm over a time-varying network and to a time-varying network 
with random link failures. 

The work in this paper is related to the literature on ergodicity of random models. A 
discussion on the ergodicity of deterministic (forward and backward) chains can be found 
in [33]. The earliest occurrence of the study of random models dates back to the work 
of Rosenblatt [32], where the algebraic and topological structure of the set of stochastic 
matrices is employed to investigate the limiting behavior of the product of independent 
identically distributed (i.i.d.) random matrices. Later, in [22], [23], [TO] , such a product is 
studied extensively under a more general assumption of stationarity, and a necessary and 
sufficient condition for the ergodicity is developed. In [7J EI] the class of i.i.d. random 
models with almost sure positive diagonal entries were studied. In particular, in [M] it 
has been showed that such a random model is ergodic if and only if its expected chain is 
ergodic. Later, this result has been extended to the stationary ergodic models in [35]. 

Unlike the work on the i.i.d. models or stationary processes in [321 [221 [231 [12], [311 135] . 
the work in this paper is on independent random models that are not necessarily inde- 
pendent. This work is a continuation of our work in [3m, where for a class of independent 
random models, we showed that the ergodicity is equivalent to the connectivity of the 
infinite flow graph of the random model or its expected model. Furthermore, unlike the 
studies that provide conditions for ergodicity of deterministic or random chains, such 
as P EE1 ESI El [121 EH E51 EE], the work presented in this paper considers the limiting 
behavior of deterministic and random models that are not necessarily ergodic. 

The main contribution of this work is in the following aspects: (1) The establishment 
of conditions on random models under which the ergodicity classes are fully character- 
ized. This result not only implies the stability of certain random dynamics, but also 
provides the structure of their equilibrium points. The structure is revealed through the 
connectivity topology of the infinite flow graph of the model. Although the model is not 
ergodic, the ergodicity happens locally for groups of indices, which are characterized as 
the vertices in the same connected component of the infinite flow graph (Theorem [6]). (2) 
The introduction and study of some perturbations (^-approximations) of chains that pre- 
serve ergodicity classes (as seen in Theorem [1]). (3) The introduction of random models 
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(class for which the ergodicity can be fully characterized by the infinite flow property 
(Theorem |4]). This class encircles many of the known ergodic deterministic and random 
models, as discussed in Section HI 

The structure of this paper is as follows: in Section [2} we present the problem of our 
interest and motivate our work by considering the limiting behavior of gossip protocol 
extended to time-varying graphs. We also define some ergodicity notions for later use. 
In Section |3j we define and investigate ^-approximations of random models which play 
an important role later on. In Section HI we introduce and study the ergodicity a class 
of random models, and we extend the infinite flow theorem of [36] to this larger class of 
models. In Section |5l we study the stability of certain random models and characterize 
their ergodicity classes. These classes are identified using infinite flow graph concept and 
analysis that combines the results of Sections [3] and HJ In Section [61 we apply our main 
result to two different random models, and we conclude in Section [7J 
Notation and Basic Terminology. We view all vectors as columns. For a vector x, 
we write Xi to denote its zth entry, and we write x > (x > 0) to denote that all its 
entries are nonnegative (positive). We use x T for the transpose of a vector x. For a vector 
x E M m , we use ||x|| p = (Y^Li \xi\ p Y^ p for p > 1 and \\x\\ when p — 2. For a matrix 
A, we write ||A|| P to denote the matrix norm induced by |||| p vector norm. We use et to 
denote the vector with the zth entry equal to 1 and all other entries equal to 0. We write 
e to denote the vector with all entries equal to 1. We write {x(k)} to denote a sequence 
x(0), x(l), ... of some elements, and we write {x(k)}k>t to denote the truncated sequence 
x(t), x(t + 1), ... for t > 0. For a given set C and a subset S of C, we write S <Z C when S 
is a proper subset of C. A set S C C with S 7^ is a nontrivial subset of C. We use [m] 
for the integer set {1, . . . , m}. We let S denote the complement of a given set S C [m] 
with respect to [m]. 

We denote the identity matrix by I. For a finite collection Ai, . . . , A T of square matri- 
ces, we write A = diag(v4i, . . . , A T ) to denote the block diagonal matrix with rth diagonal 
block being A r for 1 < r < r. For a matrix W, we write W%j to denote its (i, j)th entry, 
W l to denote its zth column vector, and W T to denote its transpose. For an m x m 
matrix W, we let Y^i<j ^ij = SjLj+i -^ or sucri a matrix and a nontrivial 

subset S C [m], we define Ws = J2ies jesO^ij + Wji). A vector v G M m is stochastic if 
v > and X^Li^i = 1- A matrix W is stochastic when all its rows are stochastic, and 
it is doubly stochastic when both W and are stochastic. We let § m denote the set of 
m x m stochastic matrices. We refer to a sequence {VF(/c)} of matrices as model or c/jam 
interchangeably. 

We write E[X] to denote the expected value of a random variable X. For an event 
we use Pr (&/) to denote its probability. For a given probability space, we say that a 
statement R holds almost surely if the set of realizations for which the statement R holds 
is an event and Pr({u; | statement R holds}) = 1. We often abbreviate "almost surely" 
by a.s. 

2 Problem Formulation and Motivation 

In this section, we describe the problems of interest and introduce some background 
concepts. We also provide an example that motivates the further development. 
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2.1 Problem of Interest and Basic Concepts 

We consider a linear dynamic system given by 

x(k + 1) = W(k)x(k) for k > t , (1) 

where {^(fc)} is a random stochastic chain, to is an initial time and x(to) G K' m is an 
initial state of the system. It is well known that, for an ergodic random chain 
the dynamics in ([[]) is convergent almost surely for any initial time to and any initial state 
x(to) (see [9]). Furthermore, the limiting value of each coordinate x;(/c) is the same, which 
is often referred to as consensus, agreement, or synchronization. In this case, the sequence 
{VF(fc)} has a single ergodic class that consists of all coordinate indices {1, . . . , m}. In 
other words, the dynamics in ([T]) is stable and the equilibrium points lie on the line 
spanned by the vector e. 

A natural question arises: what happens if is not ergodic? In particular, what 

can we say about the limiting dynamics of the coordinates Xi(t)l What can we say about 
the stability and the equilibrium points of the dynamic system (CQ)? Can we determine 
the ergodicity classes based on the properties of the matrices W{k)1 Our motivation in 
this paper is to answer these questions. To do this, we first investigate these questions 
for a more structured dynamic, namely the gossip algorithm on a time-varying network. 
Then, we show that the results are in fact applicable to a more general class of random 
dynamics. 

We next formalize several notions related to random chains. Let (f2, J 7 , Pr(-)) be a 
probability space and let Z + be the set of non-negative integers. Let W : Q x Z + — > S m be 
a random matrix process such that Wij(k) is a Borel-measurable function for all i, j G [m] 
and k > 0. We refer to such a process as a random chain or a random model. We often 
denote such a process by its coordinate representation {W(k)}. If matrices W(k) are 
independent, the model is independent. In addition, if W(k)s are identically distributed, 
{V^(A;)} is an independent identically distributed (i.i.d.) random model. 

We now introduce the concept of ergodicity. We first define it for a deterministic chain 
{A(k)}, which can be viewed as a special independent random chain by setting Q = {ui}, 
T = {{w},0}, Pr({uj}) = 1 and W(k)(u) = A(k). Then, the dynamic system in Eq. flf) 
is deterministic and we have the following definition. 

Definition 1. A chain {A(k)} is an ergodic chain or an ergodic model if \imk^oo{xi{k) — 
Xj(k)) = for any i,j G [m], any starting time to > and any starting point x(to) G M m . 
The chain admits consensus if the above assertion is true for to = 0. 

Note that, for a random model {W(fc)}, we can speak about subsets £ and C of Q on 
which the ergodicity and consensus happen, respectively. These sets are given by 

£ = n?° =0 [P\ r [ l =l {uj | lim (xi(k) - Xj{k)) = for all i,j G [m], x(t ) = e e }) , 

C = H™ =1 {u | lim (xi(k) — Xj(k)) = for all i,j G [m],x(0) = e^}. 

The scalars Xi(k) —Xj(k) are random variables since Wij(k) are Borel-measurable, so that 
£ and C are events (see [36j for a discussion on why it suffices to consider only x(0) = ee). 
The random model is ergodic (admits consensus) if the event £ (C) happens almost surely. 
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The ergodicity of certain random models is closely related to the infinite flow property, 
as shown in [36j EZ]- We will use this property, so we recall its definition below. 

Definition 2. (Infinite Flow Property) A deterministic chain {A(k)} has infinite flow 
property if YlT=o As(&) = 00 f or an V nonempty S C [m]. A random model {W(k)} has 
infinite flow property if it has infinite flow property almost surely. 

As in the case of consensus and ergodicity events, the subset of Q over which the 
infinite flow happens is an event since Wij(k)s are Borel- measurable. We denote this 
event by JF. 

In our further development, we also use some additional properties of random models 
such as weak feedback property and a common steady state in expectation, as introduced 
in [36]. For convenience, we provide them in the following definition. 

Definition 3. Let {W(k)} be a random model. We say that the model has: 

(a) Weak feedback property if there exists 7 > such that 

E[W\k) T W j {k)] > 7(E[W^(fc)] + E[Wji(k))) for all k > and i ^ j, i,j G [to]. 

(b) A common steady state n in expectation if 7r T E[W(k)] = n T for all k > 0. 

Any random model with Wa(k) > 7 > almost surely for all k > and i G [m] has 
weak feedback property. Also, the i.i.d. models with almost sure positive diagonal entries 
have weak feedback property, as seen in [35]. As an example of a model with a common 
steady state 7r in expectation, consider any i.i.d. random model {W(k)}. Another example 
is a model where each W(k) is doubly stochastic almost surely, for which we have n = — e. 

With a given random model, we associate an undirected graph which we refer to as 
infinite flow graph. We define this graph as a simple undirected graph with links that 
have sufficient information flow (simple graph is a graph without self-loops and multiple 
edges) . 

Definition 4. (Infinite Flow Graph) The infinite flow graph of a random model {W(k)} 
is the graph G°° = ([m],£°°), where {i,j} G S°° if and only if'^2'k =0 (Wij(k) + Wji(k)) = 00 
almost surely. 

The infinite flow graph has been (silently) used in [3E] mainly to establish the ergodic- 
ity of a certain deterministic chains. Here, however, we make use of this graph to establish 
ergodicity classes for certain independent random chains. In particular, as we will see in 
the later sections, the infinite flow graph and its connected components play important 
role in identifying the ergodicity classes of the model. In the sequel, a connected compo- 
nent of a graph will always be maximal with respect to the set inclusion, i.e., it will be 
the connected subgraph of the given graph that is not properly contained in any other 
connected subgraph. 

2.2 Infinite Flow Graph and Gossip Algorithm on Time- Varying 
Network 

To illustrate the use of the infinite flow graph in determining the ergodicity classes of 
a random model, we consider a gossip algorithm over a time-varying network that is 
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discussed in [36j as an extension of the original gossip algorithm [U [5] . We investigate 
the algorithm for the case when the underlying network is not connected. We assume that 
there are m agents that communicate over an undirected graph G = ([m],E) with link 
set E. The agent communication at any time k is determined by a matrix P(k) such that 
Y2i<j Pij(k) = 1- The value Pij{k) is the probability that the link {i, j} G E is activated, 
independently of the link realizations in the past, and Pij(k) = if {i, j} G" E. When 
the link {i,j} is activated at time k, agents i and j exchange their values and update as 
follows: 

Xi(k + 1) = - (xi(k) + Xj(k)) for t = i,j, 

while the other agents do not update, i.e., x^ik + 1) = xg(k) for I ^ i,j. Here, Xj(0) is 
the initial value of agent i. Thus, we have a dynamic model of the form (TjQ), where 

W(k) = I - -fa - ej)fa - ej) T with probability Py(fc). (2) 

By looking at the connected components of the infinite flow graph of the model 
we can characterize the limiting behavior of the dynamics {x(k)}. Specifically, let G°° be 
the infinite flow graph of the model. Let S\ , . . . , S T C [m] be the connected components 
of G°°, where r > 1 is the number of the components. We have the following result. 

Theorem 1. Consider the time-varying gossip algorithm given by (j2J). Then, for any 
initial vector x(0) G M m ; the dynamics {x(k)} converges almost surely. Furthermore, we 
have lim fc _ i>00 (xj(A;) — Xj(k)) = almost surely for i,j G S r and r G [r]. 

The proof of Theorem [1] relies on the infinite flow theorem, which was established 
in [36] and it is provided below for easier reference. We use the theorem to derive a 
special consequential result and, then, we proceed with the proof of Theorem [TJ 

Theorem 2. (Infinite Flow Theorem) Let a random model {W(k)} be independent, and 
have a common steady state n > in expectation and weak feedback property. Then, the 
following conditions are equivalent: 

(a) The model is ergodic. 

(b ) The model has infinite flow property. 

(c) The expected model has infinite flow property. 

(d) The expected model is ergodic. 

When the conditions of Theorem [2] are satisfied in almost sure sense, i.e., they hold 
for almost all sample paths of the random dynamics, we obtain the following result. 

Corollary 1. Let {W(k)} be a random model. Suppose that ir T W{k) = ir T almost surely 
for a vector ix > and all k>0. Also, suppose that W^k^W^k) > ■y(W ij (k) + Wji(k)) 
almost surely for some 7 > and for all k > and i,j G [m] with i 7^ j . Then, the 
infinite flow and the ergodicity events coincide almost surely, i.e., S = & almost surely. 
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We are now ready to present the proof of Theorem [TJ 

Proof. Let m r = \S r \ be the number of vertices in the connected component S r of G°°. 
Without loss of generality assume that the vertices are ordered such that Si = {1, . . . , a\}, 
S2 = Wi + 1, • • • , 02}, ■ • ■ , S T = {a T -i + 1, . . . , a T } where m r = a r — a r _i and a = < 
a\ < a,2 < • ■ ■ < a T -i < a T = m. By the Borel-Cantelli lemma [TT], page 46, for almost 
all oj G Q there exists N(u) < 00 such that no communication link will appear 

between two different components S a and Sp for a 7^ (3 and a, j3 G [r] for any time 
k > N(u). Therefore, for almost all u G f2, the chain {W(k)}(u) can be written in 
the form W(k)(u) = diag(W^(k)(u), . . . ,W^(k)(u)) for k > N(u) where W^(k)(u>) 
are m r x m r stochastic matrices. From the dynamic system perspective, this means that 
after time N(u)), the dynamics {x(k)} driven by {^(/c)} can be decoupled into r disjoint 
dynamics {x^ r \k)}, where {x^ r \k)} C M mr is governed by {W^ r \k)(uj)}. By the Borel- 
Cantelli lemma [TT] . page 46, the models {W^ r \k)(oj)}k>N(uj) have infinite flow property 
for r G [r]. Now, for every cu and each r G [r], the deterministic model {W^(k)(uj)}k>N(uj) 
is doubly stochastic and W^\k){u) > | for all i G [to] and fc > 0. Therefore, by 
Corollary [U the individual chains are ergodic almost surely. Hence, starting from any 
x(0) G M m , we have x(N(u)) = &(N(u), 0)x(0) From time N(u), the m r coordinates 
of x(A;) that belong to S r evolve by the chain {W^ r \k)}. Since each chain {W^ r \k)} 
is ergodic, it follows that limjfc-^. 00 (xi(fc) — Xj{k)) = for i,j G S r and r G [r]. Note 
that although the time N(u) is a random (stopping) time, the sets S r are deterministic. 
Q.E.D. 

Theorem [TJ states that any two agents in the same connected component of G°° will 
consent for any initial point x(0) G M m . In the upcoming sections, we develop tools 
to provide results similar to Theorem [TJ but for a larger class of random models. In 
the development, we use some refinements of the ergodicity notion, as discussed in the 
following section. 

2.3 Mutually Weakly Ergodic and Mutually Ergodic Indices 

Departing from the ergodicity and moving toward the ergodic groups of a model, we make 
use of some refinements of the ergodicity concept, as given in the following definition. 

Definition 5. Let {A(k)} be a deterministic chain, and let {x(k)} in ([TJ) be driven by 
{A(k)}. We say that: 

(a) Two indices i,j G [to] are mutually weakly ergodic indices for the chain if\im.k^,oo{xi{k) — 
Xj(k)) = for any initial time to > and any initial point x(t ) G M. m . We write 

i j when i and j are mutually weakly ergodic indices for the chain {A(k)}. 

(b) The index i G [to] is an ergodic index for the chain if linn^oo Xi(k) exists for any 
starting time t Q > and any initial point x(t ) G M. m . The chain {A(k)} is stable 
when each i G [to] is an ergodic index. 

(c) Two indices i,j G [to] are mutually ergodic indices if i and j are ergodic indices and 
i <t^a j for any initial time to > and any initial point x(t ) G M. m . We write i j 
when i and j are mutually ergodic indices for the chain {A(k)}. 
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The relation -h-a is an equivalence relation on [to] and we can consider its equivalence 
classes. We refer to the equivalence classes of this relation as ergodicity classes and to the 
resulting partitioning of [to] as the ergodicity pattern of the model. 

Definition [5] extends naturally to a random model. Specifically, if any of the properties 
in Definition [5] holds almost surely for a random model {W(k)}, we say that the model 
{W^/c)} has the corresponding property. In the further development, when unambiguous, 
we will omit the explicit dependency of the relation -h- and -v=> on the underlying chain. 

For a random model {VF(A;)}, the set of realizations for which i -h- j (or i j) is a 
measurable set; hence, an event. When the model is independent, these events are 

tail events and, therefore, each of these events happens with either probability zero or one. 
Hence, for an independent random model we write % <f-» j when Pr (i j) = 1 

and i jfc j when Pr (i <r> j) = 0. Analogously, we define i <^ j and i j. Thus, the 
ergodicity pattern of any independent random model is well-defined. 

In the light of Definition and the above discussion, by recalling the definition of the 
ergodicity (Definition [T]), we see that a chain is ergodic if and only if its ergodicity class is 
a singleton or, equivalently, its ergodicity pattern is {[to]}. Furthermore, we can interpret 
Theorem [1] as follows: for the gossip algorithm of Eq. we have i <=$w j if i and j belong 
to the same connected component of the infinite flow graph G°°. We prove in Section [5] 
that this result holds for any random model satisfying the conditions of the infinite flow 
theorem (Theorem [2]). We actually show a stronger result stating that i <=>w j if an d only 
if i and j belong to the same connected component of the infinite flow graph G°°. We also 
prove that % j if an d only if i j, where i <=>\y j is the relation for the expected 
chain {E[W(fc)]}. To show these results, we use the concept of ^-approximations and 
their properties (developed in Section [3]), and the concept of ^# 2 -class of random chains 
with their basic properties (established in Section HJ). 

3 Approximation of chains 

Here, we consider an approximation of chains that preserves ergodicity classes for the 
indices of the chains. This approximation plays a key role in our study of non-ergodic 
chains in Section [51 In what follows, we say that two chains {^(A;)} and {U(k)} have 
the same ergodicity classes if there exists a bijection 9 : [to] — > [to] between the indices of 
{W(k)} and {U(k)} such that: 

(a) i <-+w j if and only if 9(i) H[/ 9(j), and 

(b) i is an ergodic index for {H^(/c)} if and only if 9(i) is an ergodic index for {U(k)}. 

When 9 is a bijection, then the indices of one of the chains can be permuted according to 
the bijection 9, so that the bijection 9 can always be taken as identity. We assume that 
this is the case for the rest of the paper. 

We want to determine a perturbation of a chain that does not affect the ergodicity 
classes of the chain. It turns out that a perturbation that is small in £i-norm has such a 
property. To formally set up the framework for this development, we next introduce the 
concept of ^-approximation. 
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Definition 6. A deterministic chain {B(k)} is an l\- approximation of a chain {A(k)} if 
YlkLo \ A ij( k ) ~ B ij( k )\ < oo for all i,j £ [to]. 

As an example of such chains, consider two models {A(k)} and {B(k)} that differ only 
at finitely many instances, i.e., there exists some time t > such that W(k) = U(k) for 
all k > t. Since every entry in each of the matrices A(k) and B(k) is in the interval [0, 1], 
it follows that YlT=o — < t. Hence, the two models are ^-approximation 

of each other. We will use such an approximation for random models (in the proof of 
Theorem [6]), so we need to extend this notion to random models. 

Definition [6] extends to random chains by requiring that ^-approximation is almost 
sure. Specifically, a random chain {U(k)} is an ^-approximation of a random chain 
{^(/c)} if J2T=o — Uij(k)\ < oo almost surely for all i,j £ [m]. 

We have some remarks for Definition [6j First, we note that ^-approximation is an 
equivalence relation for deterministic chains, since the set of all absolutely summable se- 
quences in M is a vector space over R. This is also true for independent random chains 
and {U(k)} that are adapted to the same sigma-field. In this case, we have 
Sfclo l^ij(^) — Uij(k)\ < oo for all i,j £ [to] with either probability zero or one, due to 
Kolmogorov's 0-1 law ([TTj. page 61). Thus, {VK(/c)} and {U(k)} are ^-approximations 
of each other with either probability zero or one. Second, we note that there are alter- 
native formulations of ^-approximation. Since the matrices have a finite dimension, if 
Y,kLo ~~ Bij(k)\ < oo for all i, j £ [to], then YlkLo WMfy ~ B(k)\\ p < oo for any 

p > 1. Thus, an equivalent definition of ^-approximation is obtained by requiring that 
Er=o W A (k) - B(k)\\ p < oo for some p > 1. 

To illustrate how we can construct an ^-approximation, we consider the time-varying 
gossip model of Section [2j Let {VF(A;)} be the chain of the model as given in Eq. (j2J), 
and let G°° be its infinite flow graph. Assume that the connected components of G°° are 
Si, ... , S T . Now, define the approximate gossip model {U(k)} as follows: 

, . J W(k) if link {i,j} is activated at time k with i,j £ S r for some r, , . 
1 I otherwise. 

Basically, in the approximate gossip model, we cut the links between the agents that 
belong to different connected components of G°°. In this case, by the definition of the 
infinite flow graph we have Yl'kLo l^jW — < 00 almost surely for all % and j 

that do not belong to the same connected component of G°°. In this way, we have an 
approximate dynamic consisting of r decoupled dynamics (one per connected component 
of G°°). At the same time, the original and the approximate dynamics have the same 
ergodicity classes. This will follow from the forthcoming Lemma [H which shows the result 
for a more general model. 

Now, we present Lemma [1] which establishes the main result of this section. The 
lemma states that if two chains are ^-approximations of each other, then their ergodicity 
classes are identical. 

Lemma 1. (Approximation Lemma) Let a deterministic chain {B(k)} be an i\- approximation 
of a deterministic chain {A{k)}. Then, the models have the same ergodicity classes. 
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Proof. Suppose that i j- Let to = and let x(0) £ [0, l] m . Also, let {x{k)} be the 
dynamics as defined in Eq. (pQ) by matrices {A(k)}. For any k > 0, we have 

x(k + 1) = A{k)x{k) = (A(k) - B(k))x(k) + B(k)x(k). 

Since |xj(/c)| < 1 for any k > and any i £ [m], it follows that for all A; > 0, 

||x(fc + 1) - B(k)x(k)\\oo < \\A(k) - B(fe)||oo. (4) 

We want to show that z j, or equivalently that lim^_ >00 (xi(A;) — Xj(k)) = 0. To do 
so, we let e > be arbitrary but fixed. Since {B(k)} is an ^-approximation of {A(k)}, 
there exists time N e > such that Efcljv e ll^(^) — II 00 < £■ Let {-z(A;)}fc>Ar £ be the 
dynamics given by Eq. flTJ, which is driven by {B(k)} and started at time N e with the 
initial vector z(N e ) = x(N e ). We next show that 

k 

\\x(k + l)-z(k + l)!!^ < W A (t) ~B(t) |U for all k > N e . (5) 

i=JV £ 

We use the induction on k, so we consider k = N e . Then, by Eq. (jlj), we have |ja;(iV e + 1) — 
J B(A^ e )x(A^ e )|| 00 < \\A(N e ) - B(JV e )||oc. Since z(N e ) = x(N e ), it follows that ||ar(iV e + 1) - 
z(iV e + l)||oo < WAiNj-BiNJW^. We now assume that \\x{k) -z(fc)||oo < Eti« " 
JB^Iloo for some k > N t . Using Eq. (jlj) and the triangle inequality, we have 

+ i) _ z (fc + 1)1^ = ||A(fc)x(A;) - B(k)z(k)\\ <x 

= \\(A(k) - B(k))x(k) + B(k)(x(k) - ^(fc))||oo 

< \\(A(k) - J B(A;))|| 00 ||x(A;)|| 00 + \\B(k)\U\(x(k) - z(fc))||oo- 

By the induction hypothesis and relation ^(/c)^ = 1, which holds since B(k) is a 
stochastic matrix, it follows that ||x(A; + 1) — z{k + l)||oo < Et=jv \\ A (f) — ^(Olloo, thus 
showing relation (151) . 

Recalling that the time iV e > is such that EfcL7v c II A (k) — -B(&)||oo < e an d using 
relation ([5]), we obtain for all k > N e , 

k 00 
\\x(k + 1) - z(k + l)||oo < Yl W A W - S WH- ^ S ~ ^(Olloo < £■ (6) 

t=N e t=N e 

Therefore, \xi(k) — Zi(k)\ < e and \zj(k) — Xj(k)\ < e for any k > N e , and by the triangle 
inequality we have \(xi(k) —Xj{k)) + (zi(k) — Zj(k))\ < 2e for any k > N e . Since i -H-b j, it 
follows that lim/ c _ 5 . 0O (2;j(/c) —Zj(k)) = and limsup^^ \xi{k) —Xj(k)\ < 2e. The preceding 
relation holds for any e > 0, implying that \imk^oo{%i{k) — Xj(k)) = 0. Furthermore, the 
same analysis would go through when t is arbitrary and the initial point x(0) £ M m is 
arbitrary with ^(O)^ 7^ 1. Thus, we have % j- 

Using the same argument and inequality ([6]), one can deduce that if i is ergodic 
index for {B(k)}, then it is also an ergodic index for {A(k)}. Since ^-approximation is 
symmetric with respect to the chains, the result follows. Q.E.D. 
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Approximation lemma states that the ergodicity classes coincide for two chains that 
are ^-approximations of each other. However, the lemma does not say that the limiting 
values of the chains have to be the same for any of the ergodic classes. Within an ergodic 
class, the limiting values may differ. In particular, consider an ergodic chain {A(k)} 
and its ^-approximation chain {B(k)}. By the the approximation lemma it follows that 
B{(k)} is also ergodic. Thus, we conclude that the class of ergodic deterministic chains 
is closed under ^-approximations. The same is true for the class of ergodic independent 
random models. 

Approximation lemma is a tight result with respect to the choice of £i-norm. In other 
words, the lemma need not hold if we consider ^-approximation with p > 1. To see this, 
consider the following 2x2 chain: 



A{k) 



1 - -JL_ _L_ 

k+2 k+2 

-JL i _ _J_ 

k+2 k+2 J 



for all k > 0. 



The chain is doubly stochastic and Aa(k) > \ for % — 1,2 and all k > 0. Thus, it has 
weak feedback property with a feedback coefficient 7 — |. Also, since Yl'kLo k+2 = 00 ' 
the infinite flow theorem (Theorem [2]), the chain is ergodic. For any p > 1, the identity 
chain {/} is an ^-approximation of {A(k)}, i.e., \Aij{k) — Iij\ p = Ylk^o (k+2)p < 00 

for i,j = 1,2. However, the chain {/} is not ergodic and, therefore, ^i-norm in the 
approximation lemma cannot be replaced by any £ p -norm for p > 1. 

We now present an important result which is a more involving consequence of Lemma[TJ 
This result relates mutual weak ergodicity of a model to the connected components in the 
infinite flow graph of the model (see Definition |4]) . The result plays a crucial role in the 
characterization of the ergodicity of some chains (in forthcoming Theorem [3]). 

Lemma 2. Let {A(k)} be a deterministic chain and let G°° be its infinite flow graph. 
Then, i -h-^ j implies that i and j belong to the same connected component of G°° . 

Proof. To arrive at a contradiction, suppose that i and j belong to two different connected 
components S, T C [m] of G°°. Therefore, T C S implying that S is not empty. Also, 
since S is a connected component of G°°, it follows that Y2T=oAs{k) < 00. Without loss 
of generality, we assume that S = {1, . . . ,i*} for some i* < m, and consider the chain 
{B(k)} defined by 

Aij(k) if i 7^ j and i,jeS or i, j G S, 

if % ^ j and i e S, j e S or i e S, j G S, . . 

Au(k) + Eees A ni k ) Hi = jeS. 

The above approximation simply sets the cross terms between S and S to zero, and adds 
the corresponding values to the diagonal entries to maintain the stochasticity of the matrix 
B(k). Therefore, for the stochastic chain {B(k)} we have 

B 2 {k) ' 
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where B\(k) and ^(fc) are respectively i* x i* and (m — z*) x (m — i*) matrices for all 
k > 0. By the assumption YlT=o-^-s(k) < °°; the chain {B(k)} is an ^-approximation of 
{A(k)}. Now, let be the vector which has the first i* coordinates equal to one and the 
rest equal to zero, i.e., itj* = Yle=i e t- Then, B(k)ili* = Ui* for any k > implying that 
i j^B j- By approximation lemma (Lemma [1]) it follows i ft a j, which is a contradiction. 
Q.E.D. 

Lemma [2] applies to independent random models as well. More specifically, for an 
independent random model {W(fc)} and its infinite flow graph G°°, Lemma [2] states: if 
indices i and j are mutually weakly ergodic, then i and j belong to the same connected 
component of G°°. 

As a special consequence of Lemma [2l we obtain that the infinite flow property is 
necessary for the ergodicity (Theorem 1 in To see this, we note that by Lemma [2J 

the ergodic classes of an independent random model {^(A;)} are subsets of the connected 
components of its infinite flow graph. When the model is ergodic, its infinite flow graph 
is connected, which implies that the model has infinite flow property. 

The converse result of Lemma [2] is not true in general. For example, let A{k) = 

^ for all k > 0. In this case, the infinite flow graph is connected while the model is 

not ergodic. In the resulting dynamics, agents 1 and 2 keep swapping their initial values 
Xi(0) and x 2 (0). 



4 Ergodicity in class ^#2 

Recall that by Lemma [2] the infinite flow property is necessary for ergodicity of indepen- 
dent random models. Also, recall Theorem [2] which states that the infinite flow property is 
also sufficient for ergodicity for independent random models characterized by a common 
steady state tt > in expectation and weak feedback property. 

In this section, we introduce a larger class of random models for which the infinite flow 
property is also sufficient for ergodicity. This larger class, termed encompasses any 
independent random model that can be obtained as an ^-approximation of a model with 
a common steady state n > in expectation and weak feedback property. As a result, 
the new class includes models that do not necessarily have a common steady state n > 
in expectation. This class and its properties, which may be of interest in their own right, 
provide important pieces for the development of the main result in Section [51 

Next, we formally introduce the class of random models. 

Definition 7. An independent random model {W(k)} belongs to the class ^#2 if the 
dynamic system is such that for any t > and any x(t ) e M. m , 

00 

^2Y ; H ii (k)E[(x i (k)-x J (k)) v \ <oo, (8) 

k=to i<j 

where H(k) = E[W T (k)W(k)] . 
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The relation in Eq. (j5J) can be loosely interpreted as a requirement that the "total ex- 
pected variation" of the entries in x(k) over time is finite, where at each time the variation 
of the entries in x(k) is measured by the quantity Yli<j Hij{k)E[(xi{k) — Xj(k)) 2 ]. How- 
ever, relation ([8]) is more involving than this simple interpretation since Y2i<j Hij(k)E[{xi(k) — Xj(k)) 2 } 
need not capture actual variations in x(k), as some of the quantities Hij(k) may be zero. 

Note that, when a model has a common steady state n > in expectation, then, 
almost surely we have ([36], Theorem 5): 

oo 

^^HijiWxiik) -Xj(k)) 2 < oo, 

fc=0 i<j 

where H(k) = E [W T (k)diag(n)W (k)] . Thus, min ie[m] ^ E[W T (k)W(k)] < H(k), imply- 
ing that any independent random model with a common steady state n > belongs to 
the class . We observe that the class ^2 includes chains that do not necessarily have a 
common steady state 7r > in expectation. To see this, consider the class of deterministic 
chains {A(k)} that satisfy a bounded-connectivity condition and have a uniform lower- 
bound on their positive entries, such as those discussed in [381 Q5J EU [25], [26] . In these mod- 
els, the sequence d(x(k)) = max ig [ m ] Xi(k)— minj 6 [ m ] Xj{k) is (sub)geometric and, thus, it is 
absolutely summable. Furthermore, Hij(k) = [A T (k)A(k)]ij < m, which together with re- 
lation \xi(k)—Xj(k)\ < d(x(k)) for all i, j G [m], implies that Y^^ =0 ^ i< j[A T (k) A(k)]ij(xi(k) — 

x j(k)) 2 < °°5 i- e -5 ^e defining property of the ^2-class in Eq. ([HI) holds. 

What is interesting is that an ^-approximation of a chain with a common steady state 
Ti > in expectation also belongs to ^2, as shown in the following lemma. 

Lemma 3. Let {W(k)} be an independent random model with a common steady state 
71 > in expectation. Let an independent random model {U(k)} be an l\- approximation 
of{W(k)}. Then, {U(k)} G Jfa. 

Proof. Define function V(x) = ^(^-7r T x) 2 . Let D = diag(7r), H(k) = E[U T (k)U(k)] 
and L{k) = E[W T (k) DW (k)] for k > 0. Also, let x(0) G [0, l] m and {x(k)} be the dy- 
namics driven by the chain {U(k)}. Then, we have for any k > 0, 

E[V(x(k + l))|x(Jfe)] = E[x T {k + 1){D - 7X7x T )x(k + l)\x{k)] 

= E[(W(k)x(k) + y{k)) T (D - 7t7T T )(W(k)x(k) + y(k))\x(k)] (9) 
< E[V(W(k)x(k))\x(k)} + 2E[x{k + lf(D - mr T )y{k)\x{k)} , 

where y(k) = (U (k)—W(k))x(k) and the last inequality follows by positive semi-definiteness 
of the matrix D — 7m T . By Theorem 4 in [36], we also have E[V(W(k)x(k))\x(k)] < 
V(x(k)) — Yli<j Lij{k)(xi{k) — Xj{k)) 2 . Therefore, it follows that for all k > 0, 

E[V(x(k + l))\x(k)] < V(x(k)) - Lijik^Xiik) - x,{k)) 2 

i<j 

+ 2E[x(k + 1) T (D - 7T7z T )y(k)\x(k)] . (10) 

Since each U( k) is stochastic, we have x{k) G [0, l] m implying that |||/(fc)||oo < l|W / (^) — 
U(k) Hoo. The summation of the absolute values of the entries in the ith row of D — ttti t is 
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7Ti(2 — TTi) < 1 forallz. Thus, we have ||(D -7r7r T )y(^)J|oc < \\y{k)\\oo < \\W(k) -U(k)\\oo. 
By x(k) G [0, l] m for all k, we have \\x(k) \\i < m impying that xik + 1) T {D — 7T7Y T )y(k) < 
m\\W(k) — U(k)\\oo. Since U(k) and W(k) are independent of x(k), we obtain 

E[x(k + lf(D - nn T )y(k)\x(k)} < mE[\\W(k) - U(k)\\ 00 ] , 

which when combined with Eq. (fTUj) yields 

E[V(x(k + l))\x(k)} < V(x(k)) - L l3 {k){ Xl {k) - Xj {k)f + 2mE[\\W(k) - U(k)\U . 

i<j 

Note that YlT=o ^[11^(^0 — ^(^)||oo] < oo since {U(k)} is an ^-approximation of {W(k)}. 
Thus, by the Robbins-Siegmund theorem ([28], page 164), YlkLo ^i<j ^ij (k)(xi(k) — 
Xj(k)) 2 < oo almost surely. 

The last step is to show that the difference between the two sums Y^k=o "%2i<j Lij(k)(xi(k) — 
x^k)) 2 and EZoEiKjHijik^iik)-^)) 2 is finite. Since \W & (k) - U a (k)\ < \\W(k)- 
U(k)\\ 0O for any i,£ G [m], using the definitions of H(k) and we obtain 

in 

1=1 
m 

= E[*i(W ei (k) + [W(k) - U(k)] a )(W ej (k) + [W(k) - U(k)] tj )] 



i=\ 



< L^k) + E 



\\W(k) - Uik^J^MWaik) + W e ,(k) + 1) 



where 7r m ; n = minj S [ m ] 7Tj. The last inequality is obtained using the triangle inequality and 
the following 

[W(k) - U{k)\ u [W{k) - U(k)] £j < \\W(k) - U(k)\t < \\W(k) - U(k)\U 
Therefore, using relation (TTTj) . W^{k) G [0, 1], and the stochasticity of 7r, we have 

< Ly(fc) + 3E[||W(A;) - *7(fc)U • 

Therefore, 

oo 

Tlmin ^ y ^H ij {k){x i {k) - Xj(k)) 2 
k=0 i<j 

oo oo 

< EE^W^W -^( fc )) 2 + 3]T]TE[||W(A;) - ?7(*0lloo] (xi(fe) 

k=0 i<j k=0 i<j 

oo oo 

< E E L iA k ){*i{k) - Xj{k)) 2 + 3m 2 ]T E[||W(fc) - t/^U , 

fc=0 i<j k=0 

where the last inequality holds since (xi(k)— Xj{k)) 2 < 1. Since XlfcLo ^[ll^(^) — ^(^)lloo] < 
oo and 7r min > 0, and we have shown that J2T=o ^i<j Lij(k)(xi(k) — Xj{k)) 2 < oo almost 
surely, it follows that Ylk=o Si<j Hij(k)( x i(k) — Xj{k)) 2 < oo almost surely. Q.E.D. 
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We next show that, for the chains in class ^2 that have weak feedback property, the 
infinite flow is also sufficient for ergodicity. This result non-trivially extends the class of 
independent chains to which the infinite flow theorem applies. 

Theorem 3. Let \W{k\\ be a class ^#2 model with weak feedback property. Then, the 
infinite flow property is both necessary and sufficient for the ergodicity of the model. 

Proof. The necessity of the infinite flow property follows by Lemma El For the converse, 
assume that the model has the infinite flow property. Let to = and let x(0) G IR m be 
arbitrary. Then, since {VF(£;)} is in the *y#2-class, it follows 

00 

k=0 i<j 

where H[k) = E[W T (k)W(k)] . Due to the weak feedback property, we have E \W t (k) T W^( k)] > 
jE[Wij(k) + Wji(k)] for all k > 0, all i,j G [m] and some 7 > 0. Using the independence 
of the model and relation Hij(k) = E[W l (k) T W : > (k)~j, we obtain 

00 

E E [(Wy(A0 + Wji(k))(xi(k) - X] {k)f] 

k=0 i<j 



00 



E Z^O^M + Wji(k))(xi(k) - Xj (k)Y 

_k=0 i<j 



< OO, 



where the equality holds by (Wij(k) + Wji(k))(k)(xi(k) — Xj{k)) 2 > and the monotone 
convergence theorem ([13], page 50). Consequently, 

00 

E y^SWijjk) + Wji(k))(k)(xi(k) - Xj(k)) 2 < 00 almost surely. 

k=0 i<j 

Since the model has the infinite flow property, by Lemma 3 in [37], we have limfc_ i , 00 (a; max (/c)- 
Xmm(k)) = almost surely for any x(0) G M m , implying that the system reaches consen- 
sus almost surely. Since Y^k=t Si<i Hij(k)E[(xi(k) — Xj{k)) 2 ] < 00 for any starting time 
^0 > 0, by the same argument it follows that the model is ergodic. Q.E.D. 

Theorem [3] implies that the domain of the infinite flow theorem (Theorem [2]) can be 
extended for the chains with weak feedback property to a larger class of the models 
that do not require the existence of a common steady state vector 7r > in expectation. 
This is done in the following theorem. 

Theorem 4. Theorem^ applies to any random model that belongs to the class ^2 o,nd 
has weak feedback property. 

Proof. By Theorem El (d) implies (a). The implications (a) ^> (b) ^> (c) ^> (d) are true 
for any independent random model as proven in [36] (Theorem 7). Q.E.D. 

With this theorem we conclude our discussion on ergodic models. In the following 
section, we shift our focus on the models that are not necessarily ergodic. 
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5 Ergodicity Classes and Infinite Flow Graph 



In this section, we study models with a common steady state it > in expectation and 
weak feedback property that are not necessarily ergodic, which is equivalent to not having 
infinite flow property. Our goal is to investigate the limiting behavior of such models 
and, in particular, to characterize their ergodicity classes. We do this by considering the 
infinite flow graph of a model. To illustrate what our goal is, recall the gossip model of 
Section 12.21 In Theorem [T], we showed that the ergodicity classes of the model are related 
to the connected components of the infinite flow graph of the model. Here, we show that 
the same result holds for a more general independent random model. In the process, we 
use diagonal approximation of the model. 

In particular, consider an independent random model with a common steady 

state 7r > in expectation and weak feedback property. Let G°° be the infinite flow graph 
of the model. Assume that G°° has r > 1 connected components, and let Si, . . . , S T C 
[m] be the sets of vertices of the connected components in G°°. Let Si = {1, . . . , ai}, 
S2 = {ai + 1, • • • , 0-2}, ■ ■ ■ , S T = {ct T _i + 1, . . . , a T = m} for 1 < ai < . . . < a T = m, and 
let m r = \S r \ = a r — a r _i be the number of vertices in the rth component, where a$ = 0. 
Using the connected components of we define the diagonal approximation {W(k)} 
of {W(k)}, as follows. 

Definition 8. (Diagonal Approximation) Let {W(k)} be a random model. For 1 < r < r, 
let the random model {W^(k)} in M. mr be given as follows: for i,j e [m r ], 

W ( r )(k) = { W (i+ar-l)(i+a r -l)(k) + EteS r W(i+flr-l)*(*0 */ 3 = h ( U ) 
V K > 1 M / ( J+ a r _ 1 )( J+ a r - 1 )(^ ifj^i- 

The diagonal approximation of the model {W(k)} is the model {W(k)} defined by 



W(k) = diag(W (l \k) } . . . , W {r \k)) 



/ W^(k) ••■ \ 
W^ 2 \k) ■■■ 

\ ... W^(k) j 



Basically, in the diagonal approximation, the links between the connected components 
are removed. At the same time, in order to preserve the stochasticity of the matrices, 
the weights of the removed links are added to the self-feedback weight of corresponding 
agents. 

We now present Lemma H] which provides basic properties of diagonal approximation. 
The lemma shows that the coupling between the connected components is weak enough 
to guarantee that diagonal approximation is an ^-approximation. At the same time, 
the coupling within each of the diagonal submodels is rather strong, as each submodel 
possesses infinite flow property. 

Lemma 4. Let {W(k)} be an independent random model and {W(k)} be its diagonal 
approximation. Then, {W(k)} is an £i- approximation of {W(k)} . Furthermore, for every 
r = 1, . . . , t, the random model {W^'\k)} as given in Eq. (fl2"]l has infinite flow property. 
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Proof. First we show that W(k) is a stochastic matrix for any k > 0. Due to the diagonal 
structure of the matrix W(k) (Eq. (fT2|) ). it suffices to show that W^ r \k) is stochastic for 
1 < r < r. By the definition of W/ (r) 0), we have W {r \k) > 0. Also, for any i G [m r ], we 
have 

E^) = w^) + e w S ] ( k ) 

= W {l+ar _ l){l+ar _ l} (k) + W {l+ar „ l)e (k) + W ^+a r -l)l( k ) 

m 

= E H W)^) = 1 - 

i=i 

Now, let i G S r for 1 < r < r. Then, for any j ^ i, we have two cases: 

(i) If j G S r , by the definition of W(k), then Wy(fc) = W^k). Hence, |Wy(fc) - Wij{k)\ = 
0. 

(ii) If j £ S r , then W^fc) = and, thus, \Wij(k) - Wij(k)\ = W^k). 

For j = i, we have W u {k) = Wu(k) + J2^s r W v( k )- Hence ; \ W a( k ) ~ WtfWI = 
'Ej^Sr W n( k )i implying that J2T=i \ W ij( k ) ~ W<j(*OI = 2 £#s r Wy(*0- B Y summing these 
relations over all i E S r , we obtain for all r = 1, . . . , r, 

m 

E E - = 2 E E ^ 2W ^)- 

ieSr i=i teSt- jgSr 

Again, by summing the preceding inequalities over r — 1, . . . , r, we further obtain 

t m t 

E E E i ^ ^ - (k) i < 2 e (*) • 

r=l ig5 r i=l r=l 

Note that £U £ ie& Ejli \W l3 {k)-W l3 {k)\ = £\. g[m] |W«(Ar)-Wy(fc)|. Since S 1 , ...,S T 
are the sets of vertices of the connected components of G°°, it follows that YlT=o YH=i Ws T (k) < 
oo almost surely. Therefore, by combining the above facts, we conclude that 

oo m 

E E \ W v( k ) - WiA k )\ < a.s., 

k=0 i,j£[m] 

which proves that {W(k)} is an ^-approximation of {W{k)}. 

To prove that each submodel {W^\k)} has infinite flow property, let V C S r be 
nonempty but arbitrary. Since S r is the set of vertices of the rth connected component of 
G°°, there is an edge G S°° such that i G V and j G V. By the definition of W^(k), 
for i r = i — a r _i and j r = j — a r _i, we have w£j r (k) + W^j r (k) = Wij(k) + Wji(k). Since 
{i, j} G £°°, it follows 

oo oo 
fc=0 fc=0 

thus showing that the infinite flow graph of {W^ r \k)} is connected. Hence, {W^ r \k)} 
has infinite flow property. Q.E.D. 
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Lemma IH together with approximation lemma (Lemma [T]), provides us with basic 
tools for our study on the relations between the ergodicity classes and the infinite flow 
graph. Having these lemmas, we are now ready to characterize these relations for a certain 
random models. 

Recall that in Lemma [2} we showed that if i and j are mutually weakly ergodic, then 
i and j belong to the same connected component of the infinite flow graph. Since mutual 
ergodicity is more restrictive than mutual weak ergodicity, Lemma |2]is valid when i and j 
are mutually ergodic. The existence of models for which the converse statement holds is 
ensured by Theorem dj which shows that the extended gossip model is one of them. The 
following theorem provides a characterization of these models in a more general setting 
than the gossip. 

Theorem 5. Let {W(k)} be an independent model with a common steady state n > 
in expectation and weak feedback property. Then, i j if and only if i and j are in the 
same connected component of the infinite flow graph G°° of the model. 

Proof. Since mutual ergodicity implies mutual weak ergodicity, the "if" part follows from 
Lemma [2l To show the "only if" part, we use two ^-approximations successively. We pro- 
ceed through the following steps to prove this result. First, in order to have weak feedback 
property, we construct an ^-approximation of the diagonal approximation of the model 
Next, we prove that the resulting chain has weak feedback property. Finally, 
complete the proof by making use of the results developed in the preceding sections. 
Approximation: Let G°° have r connected components, and let Si,...,S T be the vertex 
sets corresponding to the connected components of G°°. Let 7r min = minj g [ m ] 7^ > 0. 
Consider the diagonal approximation {W(/c)} of {VT(fc)} with W^ r '(k) defined as in 
Eq. (pED for r G [r]. Let M(k) = E max iJeH \ Wij{k) - Wij{k)\ . Since {W(k)} is an 

^-approximation of {W(k)}, we have YlT=o M(k) < oo, implying that lim^oo M(k) = 0. 
Thus, there exists N > such that M(k) < f^f for all k > N. Now, let U(k) = I for 
k < N and for k > N, 

U {r \k) = (1 - d(k))W {r \k) + M e ("»r) e (mr)T 

m r 

where d(k) = ^-M{k) for k > and e^ m ^ G M mr is the vector with all entries equal to 
1. Note that ± e (™r) e {m r )T 

is a stochastic matrix. Since M(k) < for k > N, we have 
d(k) G [0,|], thus implying that U {r \k) is a convex combination of stochastic matrices 
and, hence, stochastic. 

Since Y^k=Q ^(^) < oo it follows J2h=Q M(k) < oo, thus implying that the model 
{U^(k)} k > N is an i x -approximation of {W^ r \k)}k>N- Since the entries of each matrix 
W^(k) are in [0, 1], changing finitely many matrices in a chain cannot change infinite flow 
properties. Thus, {U^(k)} is an ^-approximation of {W^ r \k)} and the model {U(k)} 
with matrices defined by U{k) = diag(t/ (1) (A;), . . . , U {r \k)), k > 0, is an £i-approximation 
of {VF(A;)}. By Lemma HI {^(A;)} is an ^-approximation of the original model 
and, therefore, {U(k)} is an ^-approximation of {W(k)}. 

Weak Feedback : Now, we show that the model {U(k)} has weak feedback property with 
feedback coefficient £ = |min(7, ^f^). For k < N, we have U(k) = I which has weak 
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feedback property with coefficient 1. So we consider U(k) for an arbitrary k > N. Let 
r G [r] be arbitrary and let Q = U^(k) to keep notation simple. For i,j G [m r ] recall 
that their corresponding indices in [m] are given by i r — i + a r — 1, j r — j + a r — 1. 
Also, recall that VF S denotes the sth column vector of a matrix VK. Using this, for any 
i,j G [m r ] with i ^ j we have: 



J ' = f (1 - d(k))W {r)i (k) + ^l e {m A ({I - d(k))W {r)j 



m r 



> (1 - d{k))\W^\k)) T W^{k) + (1 S^l e^'^^^ffc) + W^{k)Xl?>) 

m 

where in the last inequality we use 1 — > and — > — . Since e^^W^^A;) = 
e T W ir (k) it follows that 

m mm 

> — e T V^(A;) - max |Wfy(]fc) - 

m i'J'e[m] 

where the inequality holds by the stochasticity of — e. Therefore, 



> E 



'/?! 



max \Wi'f(k) — Wt>j'(k)\ 

i,'j'e[m] 



m 



-TTi - M{k), 



where the equality follows from n being a common steady state in expectation for 

and the definition of M(k). Similarly, we have E ^e^ T W^ j (k) > ±pj-M(k). Taking 

the expectation of the both sides in Eq. (Tl3l) and using the preceding inequalities, we 
obtain 



E[Q iT Q j ] > (1 - d(k)) 2 E (W ir (k)) T W jr (k) + (1 - d(k))d(k) 
> (1 - d(k)) 2 E\(W lr {k)) T W jr {k)] + (1 - d(k))d(k) 



TTi + TTj 



m 



2M(A;)J 



m 



(14) 



7TT, 



2m 



which holds by M(k) < f^jf < ^ for A; > N and 7Ti, 7Tj > 7r min for any [m r ]. 
Since W eir (k) > Wu r (k) - max^ \ Wij(k) - Wij(k)\ for all £ G [m], it follows that 

(W^(Jfc)) T W" v (lfc)l > E[(l^(fc)) T V^ v (fc)]-2mM(A;) = E[(W ir (k)) T W jr (k)] —d(k) 

where in the last equality we use 2mM(k) = d(k) 1 ^-, which follows from ^-M(k) = 
d(k). Using the above relation in Eq. (jl4|) . we have 

E\Q iT Q' J ] > (1 - d(k)) 2 E\(W ir (k)) T W^(k)] - (1 - d(k)) 2 d(k) — + (1 - d(k))d(k) — 
L J L J 2m m 

Since (1 - d{k)f < 1 - d(Jfe) it follows 

E[Q tT Q J ] > (1 - d(fc)) 2 E[(lT'-(A;)) T IU>(A;)] + (1 - d(k))d(k) 



7T,, 



> (1 - d(fc)) 2 7 E[lU v> (fc) + W jrir (k)\ + (1 - d(fc))d(fc) 



2m 

Tfmin 



2m 



(15) 
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where the last inequality follows by weak feedback property of {VF(A;)}. 

Since i r , j r G S r and % r ^ j r , by the construction of W(k), we have W ir j r {k) = Wi r j r (k). 
Hence, E Q„ ■ O r = (l_d(fc))(E[W ipir (A;)] + E[W ipip (A;)]) + ^. By combining this with 
Eq. ffT5]) . we have 



E [Q iT Q j ] > E^ + Q,] 



2d{k)\ 
m r J 



d{k))d{k) 



2m 



= (1 - d(fc)) 7 E[Q 4J + + (1 - d(k))d(k) f-^L + ^ 

Without loss of generality, we may assume 7 < ^J 1 ^, otherwise, 7' = < 7 would be a 
feedback coefficient for {VF(A;)} and the arguments hold for 7'. But for 7 < Z ^ L , we have 

-£ + 1? > o- Thus, 

E[Q iT g j '] >|E[g 4J + Q^, 

which follows from (f(/c) < |. Note that we defined Q = U^ r \k) where r G [r] and k > N 
was arbitrary. Hence, each of the decoupled random models {U^ r \k)} has weak feedback 
property with feedback constant £ = |min(7, ^f). 

Last Step: Let x(0) G K' m and {x(k)} be dynamics resulting from the chain {U(k)} accord- 
ingtoEq. ©. By Lemma[3]{[/(£;)} G uf 2s implying that EfeloEi<j L *;(*0 E [Oi(&) ~ ^-(A:)) 2 ] < 
00 almost surely, where L(k) = E[U T (k)U(k)] . Hence, for any r G [r], 

00 

^ ]T L^(fc)E[(x i (fe)-x i (fc)) 2 ] <oo. 

k=0 i<3 
i,jeSr 

Due to the diagonal structure of U(k), we have: 

(a) x(k) = (x^(k), ■ ■ ■ ,x^(k)), where x\ r \o) = x ir (0), so {x^(k)} are the sequences of 
random vectors in M. mr driven by the individual chains {U^(k)}. 

(b) For any i,j G [m r ] and any r G [r], recalling that ^ r = I + a r _i 



UuMUtiM 



te[m] 



In view of the above observations, the random dynamics in M. m induced by {U(k)} G ^2 
decomposes into r random dynamics in R mi , . . . , M. mT induced by {U^(k)}, . . . , {U^ T \k)} 
all of which belong to class ^#2- Each model {W^ r \k)} has infinite flow property so its 
•^-approximation also has infinite flow property. Furthermore, as we showed, each model 
{U^(k)} has weak feedback property. Hence, by Theorem [U {U^(k)} is an ergodic 
chain for any r G [r], which implies that i j for any i,j G [m r \. Moreover, since 
U(k) = diag(U^ (k), . . . , U^ rS, (k)), it follows i r <^ j r in {[/(A;)}. By approximation lemma 
(LemmadJ), we have i' j 1 in the original chain {VF(&)} if and only if i! ^ j' in {U(k)}, 
which is in turn true (by the structure of U(k)) if and only if i',f G S r for some r G [r]. 
Q.E.D. 



20 



Theorem [5] implies that the model satisfying the conditions of the theorem is stable. 
Furthermore, it shows that the ergodicity classes of such a model can be fully characterized 
by considering the connected components in the infinite flow graph of the model. 

Our next result further strengthens Theorem [5] by showing that this theorem also 
applies to an ^-approximation of a model that satisfies the conditions of the theorem. 
Furthermore, such an ^-approximation and its expected model have the same ergodicity 
classes. 

Theorem 6. (Extended Infinite Flow Theorem) Let an independent random model {W{k)} 
be an li- approximation of an independent random model with a common steady state n > 
in expectation and weak feedback property. Let G°° be the infinite flow graph of {W(k)} 
and G°° be the infinite flow graph of the expected model {W{k)}, where W{k) = E\W(k)]. 
Then, {W(k)} is stable almost surely and the following statements are equivalent: 

(a) i ^ w j. 

(b) % ^ w j. 

(c) i and j belong to the same connected component of G°° . 

(d) i and j belong to the same connected component of G°° . 

Proof. Since is independent, (a) implies (b) by the dominated convergence the- 

orem ([TT] page 15). By Lemma El (b) implies (c). Since < Wij(k) < 1 and the model 
is independent, by Kolmogorov's three series theorem ([7], page 63), Y^k=oWij(k) < 00 
holds a.s. only if J2kLo < oo, so (c) implies (d). Finally, by Theorem [5j (d) and 

(a) are equivalent. Q.E.D. 

By Theorem [61 we have that any dynamics driven by a random model satisfying the 
assumptions of the theorem converges almost surely. This, however, need not be true 
if either 7r > or week feedback assumption of the theorem is violated, as seen in the 
following examples. 

Example 1. Let matrices W(k) be given by 



W{k) 



1 

ui(k) u 2 {k) u 3 (k) 
1 



where u(k) = (ui(k), u 2 (k), u 3 (A;)) are i.i.d. random vectors distributed uniformly in the 
probability simplex ofM. 3 . Then, starting from the point x(0) = (0,|,1) T , the dynamics 
will not converge. This model has infinite flow property and satisfies all assumptions of 
Theorem^ except for the assumption n > 0. 

Example 2. Consider the random permutation model. Specifically, let W(k) be the i.i.d. 
model with W(k) randomly and uniformly chosen from the set of permutation matrices in 
]R m . Starting from any initial point, this model just permutes the coordinates of the initial 
point. Therefore, the dynamic is not converging for any x(0) that lies outside the subspace 
spanned by the vector e. The model has infinite flow property and has the common steady 
state 7r = — e in expectation. However, the model does not have weak feedback property, 
since E[W\k) T W j (k)] = for i ± j while E[W^(fc)] + E[Wji(k)} > 0. 
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6 Applications 



Here, we consider some applications of Theorem [2] and its extended variant to ergodicity 
classes in Theorem EJ First, we discuss the broadcast-gossip model for a time- varying 
network and, then, we consider a link failure process on random networks. 

6.1 Broadcast Gossip Algorithm on Time- Changing Networks 

Broadcast gossip algorithm has been presented and analyzed in [21 [3] for consensus over 
a static network. Here, we propose broadcast gossip algorithm for time-varying networks 
and provide a necessary and sufficient condition for ergodicity. Suppose that we have 
a network with m nodes and a sequence of simple undirected graphs {G(k)}, where 
G(k) = (jm],£(k)) and S(k) represents the topology of the network at time k. The 
sequence {G(k)} is assumed to be deterministic. Suppose that at time k, agent i G [m] 
wakes up with probability — (independently of the past) and broadcasts its value to its 
neighboring agents Ni(k) = {j G [m] \ {i,j} G S(k)}. At this time, each agent j G Ni(k) 
updates its estimate as follows: 

Xj (k + 1) = -f(k)xi(k) + (1 - 7(fc))x i (fc), 

where j(k) G (0,7] is a mixing parameter of the system at time k and 7 G (0, 1). The 
other agents keep their values unchanged, i.e., Xj(k + 1) = Xj(k) for j (jL Ni(k). Therefore, 
in this case the vector x(k) of agents' estimates Xi(k) evolves in time according to ([IT) 
where 

W{k) = I - 7(fc) e i (e i -e i ) T with probability (16) 

jeNi(k) 

Let be the infinite flow graph of the broadcast gossip model, and suppose that 
this graph has r connected components, namely Si, ... , S T . Using Theorem HI we have 
the following result. 

Lemma 5. The time-varying broadcast gossip model of f fT6|) is stable almost surely. Fur- 
thermore, any two agents are in the same ergodicity class if and only if they belong to the 
same connected component of G£° . In particular, the model is ergodic if and only if G™ 
is connected. 

Proof. In view of Theorem [6j it suffices to show that the broadcast gossip model has a 
common steady state tt > in expectation and weak feedback property. Since each agent 
is chosen uniformly at any time instance and the graph G(k) is undirected, the (random) 
entries Wij(k) and Wji(k) have the same distribution. Therefore, the expected matrix 
E[W(fc)] is a doubly stochastic matrix for any k > 0. Since j(k) < 7 < 1, it follows that 
Wu(k) > 1 — a(k) > 7 for all % G [m] and all k > 0. When a model satisfies Wu(k) > 7 > 
for all % and k, then the model has weak feedback property with as implied by Lemma 
7 in [36]. Q.E.D. 

The above result shows that no matter how the underlying network evolves with 
the time, when the broadcast gossip algorithm is applied to a time-varying network the 
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stability of the algorithm is guaranteed. In fact, we can provide a characterization of 
the connected components S r for the infinite flow graph By Theorem [6j it suffices 
to determine the infinite flow graph G™ of the expected model. A link {i,j} is in the 
edge-set of the graph Gf if and only if £~ (E[Wij(fc)] + E[Wji(k)}) = oo. By (TT^, we 
have E[Wij{k)} = ±j(k) if j G N(k) and otherwise E[Wij{k)] = 0. Thus, {i,j} G G^ if 
and only if £ fc:{lj}e£(fc) 7(fe) = oo. 

Two instances of the time- varying broadcast gossip algorithm that might be of prac- 
tical interest are: (1) The case when G(k) = G for all k > 0. Then, the random model is 
ergodic if and only if G is connected and Y2T=o 7(^) = 00 • (^) The case wnen t ne sequence 
(7(/c)} is also bounded below i.e., ■y(k) G [75,7] with < 75 < 7 < 1. Then, the model is 
ergodic if and only if, in the sequence {G(k)}, there are infinitely many edges between 5" 
and S for any nonempty S C [m]. 

6.2 Link Failure Models 

The application in this section is motivated by the work in [18] where the ergodicity of 
a random link failure model has been considered. However, the link failure model in [T8] 
corresponds to just a random model in our setting. Here, we assume that we have an 
underlying random model and that there is another random process that models link 
failure in the random model. We use {^(A;)} to denote the underlying random model, 
as in Eq. (TTJ. We let {F(k)} denote a link failure process, which is independent of the 
underlying model {W(k)}. Basically, the failure process reduces the information flow 
between agents in the underlying random model {VF(A;)}. For the failure process, we have 
either Fij(k) = or Fij{k) = 1 for all i,j G [m] and k > 0, so that {F(k)} is a binary 
matrix sequence. We define the link-failure model as the random model {U(k)} given by 

U(k) = W{k) ■ (ee T - F(k)) + diag([W(k) ■ F(k)]e), (17) 

where "•" denotes the element-wise product of two matrices. To illustrate this model, 
suppose that we have a random model {^(A;)} and suppose that each entry Wij(k) is set 
to zero (fails), when Fij(k) = 1. In this way, F(k) induces a failure pattern on W(k). The 
term W{k) ■ (ee T - F{k)) in Eq. (TIT]) reflects this effect. Thus, W{k) ■ (ee T - F(k)) does 
not have some of the entries of W(k). This lack is compensated by the feedback term 
which is equal to the sum of the failed links, the term diag([W(fc) ■ F(k)]e). This is the 
same as adding Y^j^SWi^) ' to the self-feedback weight Wu(k) of agent i at time 

k in order to ensure the stochasticity of U(k). 

Now, let us define feedback property. A random model {VK(/c)} has feedback property 
if there is 7 > such that E\Wa{k)Wi-{k) + \\\ lj (k)\\\ li (k) > 'yE\W ij {k) + W jt (k)} for any 
k > and i,j G [m] with i 7^ j. In general, this property is stronger than weak feedback 
property, as proved in [55] . 

Our discussion will be focused on a special class of link failure processes, which are 
introduced in the following definition. 

Definition 9. A uniform link-failure process is a process {F(k)} such that: 

(a) The random variables {F^k) \ i,j G [m], i ^ j} are binary i.i.d. for any fixed k>0. 
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(b) The process {F(k)} is an independent process in time. 

Note that the i.i.d. condition in Definition is assumed for a fixed time. Therefore, 
the uniform link-failure model can have a time-dependent distribution but for any given 
time the distribution of the link-failure should be identical across the different edges. 

For the uniform-link failure process, we have the following result. 

Lemma 6. Let {W(k)} be an independent model with a common steady state n > 
in expectation and feedback property. Let {F(k)} be a uniform- link failure process that is 
independent of {W(k)}. Then, the failure model {U(k)} is ergodic if and only ifYlT=ofi~ 
Pk)Ws{k) = oo for any nonempty S C [m], where pk = Pr(i^(fc) = 1). 

Proof. By the definition of {U(k)} in ( fl71) . the failure model {U(k)} is also independent 
since both {VT(fc)} and {F(k)} are independent. Then, for i ^ j and for any k > 0, we 
have 

E[U tJ (k)) = E[W y (fc)(l - Fij(k))] = (1 -p k )E[W l3 (k)} , (18) 

where the last equality holds since Wij(k) and Fij(k) are independent, and E[Fy(fc)] = pk- 
By summing the relations in (Tl8|) over j ^ i for a fixed i, we obtain J2j^i^-Wij(k)] = 
(1 - Pk) J2j^i E[M^j(fc)], which by stochasticity of W(k) implies E[J7jj(fc)] = (1 - 

Pk)(l — E\Wa(k)]). Since is stochastic, it follows that 

E[U ii(k) ] =1-J] E[^(A;)] = p fc + (1 - p fc )E[Wii(A;)] . 

From the preceding relation and Eq. (I18p . in matrix notation, the following relation holds: 

E[f7(fc)] = p fc J+(l -p fc ) E [W(*)l for all k. (19) 

Since 7r is a common steady state of {E[W r (fc)]} ) from Eq.(fl9l we obtain n T E\U{k)] = 7r T , 
thus showing that tt > is also a common steady state for {U(k)} in expectation. 

We next show that U(k) has feedback property. By the definition of U(k), Ua{k) > 
Wii(k) for all % E [m] and k > 0. Hence, E[U ii (k)U ij (k)] > E[V7«(fc)t^ i (ifc)]. Since {F(A;)} 
and {W(fc)} are independent, we have 

E[Wii(fc)M*)] = E[E[Wii(fc)^(A;) | F y (fc) = 0]] = EfEfW^^Wi^fc) | F^k) = 0]] 
= (l-Pfc)E[W«(A;)Wi i (fe)]. 

A similar relation holds for E[Ujj(k)Uji(k)]. By the feedback property of {W(k)}, we have 

E[U u {k)U l3 {k) + Ujj(k)Uji(k)] > (1 -p fc ) 7 E[^(A;) + W^A;)] = 7 E[^(A;) + ^(A;)] , 

where the last equality follows from Eq. ffl9|) . and 7 > is the feedback constant for 
{W(k)}. Thus, {U(k)} has feedback property with the same constant 7 as the model 
Hence, the model {U(k)} satisfies the assumptions of Theorem [2j so the model 
{U(k)} is ergodic if and only if YlT=o ^[Us(k)] = 00 for any nontrivial S C [m]. By 
Eq. (fT9~l) we have E[Us(k)] = (1 — pk)E[Ws(k)], implying that {U(k)} is ergodic if and 
only if J2T=o(^ ~ Pk)E[Ws(k)] = 00 for any nontrivial S C [m]. Q.E.D. 
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Lemma [6] shows that the severity of a uniform link failure process cannot cause insta- 
bility in the system. When the failure probabilities pk are bounded away from 1 uniformly, 
i.e., pk < p for all k and some p < 1, it can be seen that ^^L (l — Pfe)E[Ws(A;)] = oo if 
and only if ^^1 E[M / s(A;)] = oo. In this case, by Lemma [6] the following result is valid: 
the failure model {U(k)} is ergodic if and only if the original model {^(A;)} is ergodic. 

7 Conclusion 

In this paper, we have studied the limiting behavior of time-varying dynamics driven by 
random stochastic matrices. We have introduced the concept of ^-approximation of a 
chain and have shown that, for certain chains, such approximations preserve the limiting 
behavior of the original chains. We have also introduced the class of stochastic chains 
to which the infinite flow theorem is applicable, which non-trivially extends the class of 
models originally covered by this theorem [36J. Finally, we have identified a certain class 
of independent random models that are stable almost surely. Moreover, we characterized 
the equilibrium points of these models by looking at their infinite flow graphs. Finally, we 
have applied our main result to a broadcast gossip algorithm over a time- varying network 
and to a link-failure model. 
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